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' Abstract 



\^ , We investigate performance limits of a multiple access communication system with energy harvesting 

nodes where the utility function is taken to be the long-term average channel throughput. We assume a 
causal structure for energy arrivals and study the problem in the continuous time regime. For this setting, 



we first characterize a storage dam model that captures the dynamics of a battery with energy harvesting 
and variable transmission power. Using this model, we next establish an upper bound on the throughput 
problem as a function of battery capacity. We also formulate a non-linear optimization problem to 
determine optimal achievable power policies for transmitters. Applying calculus of variation techniques, 
we then derive Euler-Lagrange equations as necessary conditions for optimum power policies in terms of 
a system of coupled partial integro-differential equations (PIDEs). Based on the block coordinate descent 
algorithm, we devise an iterative algorithm to solve these equations. We also propose a modified version 
of this algorithm for the symmetric multiple access setting in which the statistical descriptions of energy 
harvesters are identical. Along with the analysis and to support our iterative algorithms, comprehensive 
' numerical results are also obtained. 

Index Terms 

Energy harvesting, multiple access communication, iterative algorithm. 



I. Introduction 

The direct impact of energy on communication cost and lifetime has spurred significant efforts to 
manage and optimize energy consumption. In this respect, current and future state of the art technology 
has focused on harvesting energy from the environment. It is thus of paramount importance to design 
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suitable adaptive power transmission policies for these technologies. In particular, the formulation of 
power policies in energy harvesting systems depends intricately on many factors, including energy arrival 
distribution, battery capacity, quality of service, etc. Moreover, most renewable energy resources have 
unpredictable behaviour that makes the design process of optimal power policies difficult. Solar panels, 
for instance, can only scavenge sunlight during the daytime and even then, this is a function of weather 
conditions. Another example is thermoelectric generators where energy is absorbed based on random 
temperature gradients between two metal junctions. Regarding these examples, a key objective of recent 
studies is to engineer optimal transmission power polices. These studies, depending on causal or non- 
causal knowledge of future energy arrivals, fall within two major categories: offline or deterministic (for 
non-causal), and online or stochastic (for causal) energy harvesting systems. 

In the offline regime and in terms of throughput maximization, optimal power allocation for different 
communication topologies has been well studied. For instance, |[23l studies the multiple access channel 
(MAC), 11211 studies the broadcast channel, and the interference channel is studied in ['20']. In addition, 
the issue of maximizing throughput in a fading channel has been treated in Iffll . There, a directional 
water-filling algorithm is proposed. In [18], queuing aspects of the offline energy harvesting problem 
with infinite battery and buffer capacity have been considered. The authors have also suggested a greedy 
policy that in the low signal to noise ratio (SNR) regime is throughput optimal and attains minimum 
delay. In [5], a continuous time energy harvesting system with constant energy leakage rate due to battery 
imperfections is considered. Another interesting problem has been studied in |[22l where an offline energy 
harvesting problem subject to minimizing the transmission completion time is analyzed. Specifically, a 
continuous-time policy to minimize the delivery time of data packets is formulated. Among more recent 
results in the offline setting is [8] where energy cooperation in a two-hop communication system is 
considered. 

While all of these aforementioned works study the problem of offline energy harvesting, very few offer 
structural results for the online problem. As an overview of prior works in this regime, we refer the reader 
to El, US), lfT6l , and (15). In fffl an algorithm in the offline problem of throughput maximization by a 
deadline was heuristically applied to the online counterpart. The authors have also considered a dynamic 
programming solution for online policies. Nevertheless, the curse of dimensionality in the backward 
induction renders the computational cost of this approach very expensive. In |[T6l , the capacity of the 
additive white Gaussian noise channel (AWGN) under discrete-time energy arrivals and infinite battery 
capacity is characterized. Additionally, two achievable schemes based on save-and-transmit and best- 
effort-transmit are studied there. A more relevant study related to the work presented here is[ 19 "]. Therein, 



Srivastava and Koksal have investigated an optimization problem where the objective is to maximize a 
utility function subject to causality and battery constraints. More interestingly, they addressed a trade-off 
between achieving the optimum utility and keeping the discharge rate low. 

Herein, we consider the online setting with continuous time policies in which the energy release rates 
are regulated dynamically based on the remaining charge of the battery at each moment. This distinctive 
architecture naturally appeals for a different mathematical framework in terms of modelling and analysis. 
Particularly, the main tool here for modelling the interaction between battery, energy arrivals, and energy 
consumption is a stochastic process known as a compound Poisson dam model. This model was pioneered 
by Moran in 1954 [13] and studied further by Gaver-Miller [6 | and Harrison-Resnick [9]. In connection 
with this model, we explore an upper bound on the total throughput of an online energy harvesting 
system. Also in terms of achievability, we construct an optimization problem to maximize the channel 
throughput subject to an ergodicity constraint. This maximization problem turns out to be non-linear and 
analytically cumbersome. Relying on a calculus of variations approach, we subsequently find a system 
of simultaneous PIDEs as necessary conditions for an optimal power policy. We then propose a variation 
of the block coordinate descent (Gauss-Seidel) method (see [3]) to solve these equations efficiently. In 
the symmetric case, when the statistical description of all the energy harvesters are identical, we show 
that this algorithm reduces to a fixed point iteration method. Moreover, in the case of the point-to-point 
channel setting, the necessary condition further reduces to a non-linear, autonomous ordinary differential 
equation (ODE) that can be solved directly, using conventional numerical methods |[T2l . 

The rest of the paper is organized as follows. In section II, we review some background, definitions, 
and notation. Section III deals with necessary and sufficient conditions for ergodicity of the storage 
process. In Section IV, we put forth an upper bound as well as the achievability results for both finite and 
infinite storage cases, including two algorithms for the achievability part. These algorithms are then used 
to compute the numerical results in Section V. Lastly, in Section VI, we summarize our main findings 
and outline possible future directions. 

II. Preliminaries 

A. Communication model 

We consider M multiple access transmission nodes that wish to transmit their data over a shared 
communication channel. Furthermore, each transmission node is equipped with an energy harvesting 
module and a battery to capture and store arriving energy packets. Throughout the paper, we denote the 
instantaneous transmission power at time t from the k th node (k = 1,2,- •• , M ) by Pk{t). Also, to 



quantify the corresponding transmission rate of the nodes, we consider Shannon's rate function, r(x) = 
\ log 2 (l + (x/No)), where iVo denotes the noise power spectral density. In particular, Shannon's rate 
function carries the following properties and only these properties will be used in Section IV: 

• [Positivity] r(x) > for all x > and r(0) = 0. 

• [Differentiability] r(x) is a smooth function. 

• [Monotone non-decreasing] Vx > : r'{x) > 0. 

• [Concavity] Vx > : r"(x) < 0. 

Letting Rk denote the long-term average rate of the k th user, we then have the rate -region described 

by 

T 

E Rk ^ ^ lim ^ / r ( E Pk ^) ds ^ 

1 r CO ± } r\ 
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where the inequality holds for all subsets S C {1,2,- •• ,M}, and the resulting region is a poly tope 

called polymatroid. In this study, we restrict ourselves to the dominant face of this polymatroid (called 

permutahedron) that represents the total throughput of the channel. Then, the sum-rate is 

M „T M 

E^fe= lim - / r(E^W) ds. (2) 

B. Energy harvesting and storage model 

In our energy harvesting model, we allow the transmission nodes to use different techniques for 
harnessing exogenous energy. For example, while one node may collect solar energy, another node 
can use a thermoelectric generator. This mechanism is especially important for sensor networks where 
distributed terminals may measure miscellaneous targets that also feed sensors with energy {e.g. see Q). 
Mathematically, we assume that for each individual node k € {1, 2, • • • , M}, energy is replenished into 
the corresponding battery according to specific energy arrivals E\, ■ ■ ■ , where the superscript denotes 
the order of arrivals. Furthermore, the energy arrivals for node k are independent, identically distributed 
(Ltd) according to P{Ek < x} = B^{x) which occur at random arrival times denoted by T°,T^, 
The interarrival times AT™ = T™ +1 - Tg are also assumed to be i.i.d and exponentially distributed. 
Therefore, the attributed point process, Nk(t) = ^ n l^ <t ^, is a homogeneous Poisson point process 
with intensity denoted by Afe. Consequently, the total energy flow £j[ n (0,i] into node k and up to time t 
is a compound Poisson process, 

N(t) 

4 n (0,t] = E^- ( 3 ) 

i=0 



To characterize the storage model, we also need to specify the output process at each transmitter. To do 
so, let Xk(t) denote the energy stored in the k-th battery as a function of time. Then, the total energy 
expenditure until time t is 

4° ut (*) = f P k (s)ds, (4) 
J o 

= / Pk (X k (s)) ds, (5) 
J o 

where p k {-) represents the transmission power policy of the k-th transmitter, modulated by the available 
energy in the battery. Now, the storage equation in terms of the energy arrivals in © and the drift process 
in © is 

X k (t) = X k (0) + 4 n (0,t] - [ Pk (X k (s)) ds, (6) 

Jo 

where x k (0) is the initial (possibly random) battery reserve at time t = 0, and here the battery is assumed 
to have infinite capacity (X k (t) E [0, oo)). In the case that the k-th battery has a finite storage capacity, 
say L k , then X k E [0, L k ], and we can similarly characterize the following dynamics, 

X k (t) = X k (0) + 4 In (0,t] - / Pk(X k (s)) ds - Z k (t), (7) 

Jo 

where Z k (t) is M + valued process that is null at zero (Z k (0) = 0), non-decreasing, continuous almost 
everywhere, and such that f R+ (L k — X k (s)) dZ k (s) = 0. This process, known as reflection process [17], 
ensures that for any energy arrival, the storage process remains inside the boundary, i.e., X k (t) E [0, L k \. 

It is also interesting to note that the application of the structures in © and (|7|) are not limited to 
the current problem. In fact, this formulation has wide applicability in other fields of studies. Examples 
include workload modulated queues[4], water reservoir dam analysis [1], food contaminants exposure in 
bioscience [2], etc. In this paper, the ergodicity results of [1] will be used and are summarized in section 

in. 

C. Notation 

In the rest of the paper and for conciseness, we adopt several shorthand notations. In particular, [M] 
stands for {1, 2, • • • , M}. For M > 1, we define the rectangular domain A as 

A±[0,L!] x [0,L 2 ] x ■■■{0,L M ). 

Related to this, we also define the M dimensional integral by 

Li Li Lm 

J (•) dxidx 2 ■ ■ ■ dxM, 






which is represented by f A {-) dx. For all subsets S C [M], we use .4(5) to denote the projection of A 
onto the coordinates indexed by S, i.e., 

A({1,3}) = [0,L X ] x [0,L 3 ]. 

Then, Jm S \{') dx denotes integration over a subset of M) s \. Aj is also a shorthand for 

Aj = [0, L x \ ■ ■ ■ [0, Lj_i] x [0, • • • [0, L M ]. 

Finally, to avoid confusion between energy arrivals and the expectation operator, we use E[-] for the 
latter. 

III. Ergodic Theory of Storage Process 

We here summarize necessary and sufficient conditions for ergodicity of the storage process in (j6). 
Before stating the definitions regarding ergodic behaviour, we first put some mild constraints on the 
transmission policies. Particularly, for all k = 1, 2, • • • , M, 

1) VL fc > : < x k <L k ^ p k (x k ) > and p k (0) = 0, 

2) VL fc > 0, sup p k {x k ) < oo. 

0<x k <L k 

The first condition indicates that as long as there is energy in the battery, transmission continues 
(otherwise, the battery would have a minimum energy reserve that can not be consumed). The second 
condition does not permit the energy in the battery to be consumed instantly. Regarding these constraints, 
we say a policy is admissible iff it fulfills these two conditions. 

Definition 1: The hitting time, t(x), is defined as the first time that the energy level in the battery 
reaches the value of x. More specifically, 

t(x) = mf{t > : X(t) = x}. 

Definition 2: [1, pp. 290] The storage process is said to be transient, if and only if for all initial 
energy levels x(0) in the battery, we have F(X t — > oo) = 1. Alternatively, the storage process is said to 
be recurrent if and only if P[r(x) < oo|x(0)] = 1, Vx > 0, x(0) > 0. In the case of a recurrent storage 
process, it is said to be positive recurrent if it further satisfies E[t(x)|x(0) = x] < oo for one x > 
and therefore for all x > (irreducibility). Similarly, the recurrent storage process is null recurrent if 
E[r(a;)|x(0) = x] = oo for one x > and therefore for all x > 0. 

One motivation for surveying ergodic conditions is to rule out policies that result in transient and null 
recurrent battery behaviours. For example in the transient case X(t) — > oo a.s. which is unrealistic. Also, 



in the null recurrent case lim^oo ¥{X(t) < u\x(0) = x} = 0,Vx,u > which implies an unbounded 
energy reserve in the battery. 

Theorem 1: (Assmussen [1, Thm. 3.6]) The storage process {X k (t)} t >o is positive recurrent if and 
only if there exist a probability measure ir k that is absolutely continuous on (0, oo) and which may 
possess an atom at zero, TT k = TT k ({0}), i.e., 

K k {xk) = TT° k + / fk(vk) dv k , (8) 

and such that 

fk(xk) = —^{4(1 - B k {x k )) + f \l - B k (x k - v k ))f k (v k ) dv k \. (9) 
Pk{Xk) l> Jo+ J 

Furthermore, ir k is the unique stationary distribution of the process X k (t). □ 
Remark 1: The elegant proof of Assmussen for the converse part of Theorem 1 is based on an 
embedded Markov chain {X k (n)} at marked arrival times. In particular, for recurrent embedded chains, it 
is shown that any storage interval (x k , x k ), < x k < x\ is recurrent in the sense of Harris. An alternative 
proof of the converse part of Theorem 1 adopts the additional condition 9(x k ,0) = f^ k (l/p k (u)) du < 
oo,Vx k > 0. In light of this extra condition, the required time to reach the zero state in the absence 
of new arrivals from any energy level in the battery must be finite. For this constraint, it can also be 
shown that x k = is a regenerative recurrent point for the process and therefore, due to the additional 
constraint, the probability measure has a strict atom n k > at zero. 

Remark 2: As discussed in [1, pp. 297], in the finite energy case (L k < oo), the storage process is 
always positive recurrent and the probability measure is likewise governed by ([8]) and (|9]>. 

Remark 3: Due to the first constraint on the admissible power policies and in particular p k (L k ) > 0, 
there is no atom at x k = L k in the finite case, and therefore, the battery never idles with x k = L k 
(reflecting boundary). 

An interpretation for the forward equation in © can be provided in terms of level crossing theory. 
Particularly, 

fk(x k )Pk(xk) = A fc |7r^(l - B k (x k )) + j (1 - B k (x k -v k ))fk(vk) dvkj, (10) 

reflects the equilibrium condition between the rate of down crossing at level x k (the l.h.s of (fTOl ) and up 
crossing at level x k (the r.h.s of (fTOl). We can also view (U as a Volterra integral equation of the second 
kind with the kernel function Jff(x k , v k ) = 1 — B k (x k — v k ), and it can thus be solved numerically (see 

El). 



In this paper, we consider the energy arrivals {E l k }^ , k = 1, 2, • • • , M, to be exponentially distributed 
with parameter ( k . Thereby, we have 



that simplifies © to 



where 



>~T(x k ,v k ) = exp(-CfcO fc - v k )), 



A fe exp(-CfcX fc ) 

/fc(Zfc) = 7 — ; G k (x k ), (11) 

Pfe(£fc) 



G k {x k ) = ^7r + y exp(( k v k )f(v k ) dv k y (12) 

Remark 4: The storage models in © and © are memoryless, in the sense that at each time instant t, 
the power policy p k only depends on the available charge X k (t) in the battery and not the entire sample 
path {X k (s); s < t}. As an extension, we can also define a storage model with memory as follows 

X k (t) = X k (0) + 4 n (0,t}- [ Pk (X k (u);u<s) ds. (13) 

J o 

However, when the arrival process is Poisson, it can be shown that X k (t) is a sufficient statistic for an 
optimal power policy (see Appendix A). In this regard, knowledge of the entire path {X k (s); s < t} as 
an argument of p k {-) is excessive. 

IV. Bounds on Total Average Throughput 

Our objective now is to derive and develop an upper bound on the average throughput as well as 
achievable policies with good performance. In connection with our system model, we will analyze a 
MAC with 1) finite, and 2) infinite storage batteries. 

In particular, in the finite storage case, a good power policy must manage overflow in the battery 
as regular overflow causes energy waste and potentially decreases the channel throughput. To reduce 
overflow, the power policy must result in a large transmission power when the battery charge is large 
as otherwise overflow is likely to occur upon a new arrival. Nevertheless, transmitting with too large a 
transmission power when the battery happens to have large charge is also undesirable due to the concavity 
of the rate function. In other words, there is a tension between overflow and the rate at which the large 
battery charge is consumed to reduce overflow likelihood. 

To further clarify the latter point, consider an energy harvesting system with a single node (M = 1) 
in which energy E is replenished into a battery exactly every T units of time. In addition, assume that 



the transmitter sends data by using a constant transmission power P = E/(aT), a > 0. Two cases can 
now be examined: 

i) a > 1: In this case, the transmitter fails to consume the entire battery charge before the next arrival, 
and thus overflow occurs regularly. We then have 

T x r ( — ) <T x r ( — ) . (14) 



ii) a < 1: In this case, the transmitter depletes its available battery charge within aT < T of each 
arrival. From the concavity of the rate function, we have the following inequality 

aT x r ( <T x r ( ^] . (15) 



aT J ~ \T 

Here, the tension between (/) and (if) is resolved by the optimal choice of a = 1, i.e., P = E/T. 
A. An Upper Bound 

1) Finite Storage Battery: In this case Lk < oo,V/c € [M]. Then from © and due to ergodicity of 
the storage processes {^fc(t)t>o}fcli i n tne finite battery case (ref. Remark 3), we have almost surely 



M M 

(16) 



k=l k=l 

where the expectation is with respect to the stationary distribution in Theorem 1. In addition, from the 
concavity property of the rate function and Jensen's inequality, 

M M 

E[r(J> fe (X fe ))] <r(J2^\Pk(X k )\). (17) 

k=l k = l 

It thus remains to bound the mean transmission power E[p k (x k )]. This can be accomplished by integrating 
by parts as follows 

K\p k (X k )} = 7r°p fe (0) + / " Pk {x k )f k {x k ) dx k (18) 

- / Pk(xk)fk(xk) dx k (19) 

= A fc / exp(-( k x k )G k (x k ) dx k (20) 

= --^ex.p(-( k x k )G k (x k ) + — / exp(-( k x k )G' k (x k ) dx k , (21) 
Cfc o+ Cfc Jo+ 

where (a) comes from the first constraint on the admissible power policies and (b) follows from (fTTT i. 
Now from (fT2l . 

G'k( x k) = fk(xk) exp(( k x k ). (22) 



Also we note that G k (0 + ) = 7r° and 



e 



CkLk G k (L k ) = e" Cfcifc (vr° + ! k e^ Xk f k {x k ) dx k ) (23) 



> e- CfcLfe (vrO+ / k f k (x k ) dx k ) (24) 

= e -^ kLk . (25) 
Substituting (1231 ) and (|22l in (|2TT) thus leaves us 

E[ Pk (X k )} = -±(G k (0+) - e^ L *G fc (L fc )) + / / fc (x fc ) dx fc , (26) 

<^{7T° k - e-^ Lk + / " f k {x k ) dx k ) (27) 

Sfe J0+ 

= ^(i-exp(-aL fc )), (28) 



In the last step, we now use (1281) and the non-decreasing property of the rate function to characterize an 
upper bound for all L k < oo as follows 

M M . 

^R k <r(^f(l-e-^))±R uppei . (29) 

k=i k=i ^ k 

2) Infinite Storage Battery: We now take L k = oo. In this case, similar to (l2Qb we can directly 
compute, 

/■oo 

Ebfc(x fc )] = A fc / e~^ Xk G k {x k ) dx k (30) 
= Xk f e-^(7r°+ / V*'"-/fc(t>fc)*>k) (31) 



A 



o+ v Jo+ 

OO fX k 



A -7r k + X k / e Cfe ^-^)/ fe (^) dv k dx k (32) 
( = } ^vr° + A fc /°° [°° e^ v "- x ^f k (v k ) dx k dv k (33) 
= ^ k + ¥ I" fk(v k )dv k (34) 



(35) 



'o+ 

A^ 

"a' 

where in (a), we changed the order of integration due to the Fubini's theorem. Thus, for positive recurrent 
policies and when all L k = oo, we have the following upper bound 

M M . 

5>^(£#)- (36) 

k=l k=l ^ h 



B. Achievable allocation scheme 

To derive transmission power policies with good performance, we start with the ergodicity assumption 

and the definition of expectation, i.e., 

M „t M 



J2 R k = r l™ \ [ r{ P ^)) ds (37) 
k=i Jo k=i 

,. M M 

= r(^2p k (x k ))Y\_K k (dx k ) (38) 

Ja k=i k=i 

i M 



^n{{p k (x k )}tU), (39) 
where 

-ir k (dx k ) = K<5(x fc ) + f k (x k )]dx k , (40) 

and 5(x k ) denotes the Dirac delta function. We now aim to find achievable policies through the following 
optimization problem 

„ M M 

sup / r(y\p k {x k )) TT TT k (dx k ), (41) 
{■xlMxM^JA k=1 k=1 

s.t : Vk G [M] 

fk(xk) = ^7^(4 + r e ~ CkV M v ) dv )i ( 42 ) 
Pk{x k ) V J 0+ J 

< + [ " fk(xk) dx k = 1, (43) 



tt£>0, / fc (ar fc )>0, (44) 

which maximizes the overall expected throughput of the multiple access channel subject to the stationary 
probability measure constraints of the batteries. Nonetheless, tackling this non-linear optimization problem 
is challenging as the feasibility constraint in (l42l is not in an explicit form. To circumvent this difficulty, 
we use a calculus of variations approach to transform the problem into a set of necessary conditions for 
an optimal solution. As a starting point, consider the following linear mappings 

9k{x k ) = fk{x k )e^ x \ x k >0, (45) 
that converts the positive recurrent condition in (UTt into 

9k(x k ) = + / 9k(v) dv) (46) 

Pk(Xk) V J + / 

A i. 



with G k (xk) = (V^ + Jq+ g k (v) dv^j as in £[2]>. Hence, ([38]) is valid with 

{^kG k (x k )/g k {x k ) x k > 
(48) 
x k = 0, 

7r fc (cte fc ) = [7rg£(x fe ) + e^*"*^^]^*. (49) 

With this substitution, we obtain an equivalent formulation for the optimization problem in (l4~TTi-(l4"4l as 
below 

, M M 

sup / r(^2p k (x k )) Y[ K k (dx k ), (50) 
s.t : Vk G [M] 

G k {x k ) = (vr° + I " g k {v) dv), (51) 



7r° k + [ k e- (kV g k (v) dv = l, (52) 

<>0, g k {x k )>0, (53) 

where p k (x k ) and Tt k (dx k ) are according to (l48l and (|49l . 

Through the formulation in (|50ll-(l53Tl, we can show that the throughput maximization problem in (1411 - 
(l44l) is concave with respect to each coordinate over a convex feasible set. In particular, since the trans- 
formation between f k (x k ) and g k (x k ) is linear, the concavity of (l4ll- (l4"4l can be shown equivalently by 
proving the concavity of the formulation in (|50l-(l53l. To this end, suppose that {(vr^' 1 , g k {x k ))^ =1 an d 
{ (7r°' 2 , g k (x k )) } fc=1 are two arbitrary sets of optimization parameters belonging to the feasible region 
defined in gT)-©. Then for all a G [0, 1] and a = (1 - a), it readily follows that { (vr"' Q , 
also satisfies d5T|)-(|53l, where 7r°' a = an®' 1 + a7r°' 2 and g k (x k ) = ag k {x k ) + ag k (x k ) are the convex 
combinations of the densities and atoms, respectively. This proves the convexity of the feasible region 

(EB-dSS. 

Proposition 1: Let TV- and IZj be the utility functions corresponding to { (vr°' Q , g£ (xfc)) } , x , 
{K' 1 >^fc( a: fc))}fc=i' and {(^fc 2 '^^))}^! respectively, such that 

K' a )5fe(^fe)) = "(^fc \5fe(zfc)) +a(vr°' 2 ,^(x fc )), A; = j, 



7T 



0,a 
fc ' 



Then, 



n^dR^ + aiq. (54) 



Proof: The proof is relegated to Appendix B. 
Now define an ensemble of perturbation functions, {V'fcjfcli' sucn that 



0"t 



i/> h (v) dv = 0, (55) 
exp(-( k v)ip k (v) dv = 0, (56) 

o+ 

and the ip k are continuous and bounded on their domain (0,L k ] with ^fc(O) = 0. For sufficiently small 
e k > 0,k G [M], it thus follows that gl h (xk) = g k (x k ) + £ktpk(x k ) meets (|5T|)-(|53T) with the same atoms 
7r° and thus lies inside the feasibility region. Then, with e = (ei, e2, • • • , tu ), it must be true for a global 
maximum solution that 

TV < n, (57) 

where 

„ M M 

1¥= rf^^wjn^M, (58) 

J A fc=i fc=i 

and 

ir e k k (x k ) = [nlS(x k ) + e- CkXk g k (x k ) + e fc e- Cfca;fc ^ fc (x fc )](ix fc 

= TT k (dx k ) + e k e~ CkXk ilJk(xk)dx k , (59) 
and p k k {x k ) is calculated from (|48l to be, 

G k (x k ) + e fc ^fc(x fc ) x > q 
%fc(zfc) + e fe Vfc(zfc) * (60) 
x k = 0, 



with, 

4 / "^(t,) <fo. (61) 
For the moment, we assume that only the j th coordinate is perturbed; that is e k = 0, V/c ^ j. Expanding 



the right hand side of (1581 ) to first order then results in 



M 



Tl ei = / r(^p fe (x fc )) + e, 



fc=i 



dpj(xj) 



e,=0 



x ^7Tj ( dxj ) + e j e ~ ^ ^ ( x j ) dxj ] JJ Tr k (dx k ) 

k€[M]-j 

f M 

n + e i / ^(Vpfc(^fc))e Ci:E3 Vj(^)da;j FT K k {dx k ) 



(62) 



fc=i 



A 



dp j (x J/ 



dej 



e,=0 



ke[M]-j 

M 

J\ Kk{dx k ) 



k=l 



+ 0{e]). 
On the other hand, we note that 



dp?(0) 



dej 



e,=0 



since p k (0) = from (l60l lj| Therefore, 



e,=0 



S(xj)dxj = 0, 



(63) 



(64) 



(65) 



and we thus have from d63 



„ M 

K ej =11 + ej J ri^i>,A.n))<- '' / •,(.;•, Ir/./j vr fc ((ix fc ) 



A k=l ke[M]-j 



A 



dpjixj) 



de^ 



gj(xj)dxj IT ir k (dx k ) + 0(e]). 

€ ■ =0 

J fee[Af]-j 
This expansion, accompanied with inequality (T57T ) establishes the following necessary condition for a 

locally (and thus globally) optimal solution 



„ M 

I r(y'pk(xk))e c ' le *i/> j (x j )dx j TT 7r fc (dx fc ) 



+ 



fc=i fc £ [Af]_j 

M 



M££iP*(**))*?( 



e,=0 



e ijXi gj(xj)dXj ] { 7r k (dx k ) = 0, 



(66) 



fcG[M]-j 



and we have neglected the second order term O(e^). Now with abuse of notation, let 



M „ M 

E j[ r (y^Pk( x k))] = / kx^*^*)) n ^kidxk), 

k=l J Aj k=1 ke[M]-j 



(67) 



Alternatively, (0) = for all ej as the battery is empty. 



denote the expectation over all the coordinates except the j th . Then (l66l ) can be restated as 

U -dEj [r{Zk=i Pk(xk))] dp** (Xj) 

e,=0 



M 



dpj(xj) 



where we used the fact that 



dej 



e-^ i7j (x i )+E i [r(X;PfcM]e-^^0 



fc=i 



r 5r (Er=i^(^))i 9 



3 d Pj( z j) 

On the other hand, from (|60l i, we compute 



-E^r^p^))]. 



i=l 



dpj(xj) 



- ' / ^'./ //(( . r/i - 



We thus further proceed by substituting (ITO in d68), 



= 0, 
(68) 

(69) 

(70) 
(71) 



3 dPj(xj) 



II 



dp^Xj 



M 



1 ' ' ' ' k=l 

Integrating by parts, the second integral can be evaluated as follows 

M 

mXk))i r,Jr.A + p-b x mJr( 
dPjiXjj 



Pjixj) + e-t* x 'Ej [r{^2p k {x k ))] i>j{xj) d Xj = 0. (72) 



/ r-^ d ^ WVLiPkfrk))] p . (x . ) + e -^ % [r( J2 Pk (x k )))} Mxj) dx 

J L a Pj\ x j) ._, 

o fc — 1 

M 



gE j[ r (Efc=iPfc(^))] 



p^Xj-) + e-^^Ej [r( J]pfc(xfc))] 



\1/ j (Xj / 



fe=i 



+ 



3a; ,• 



^ E j[ r (E fc =iPfc(^fc))] 

<%(xj) 



Pi(^i)+e <iXi Ej[r(52pk(x k ))] 



Vj(zj) dxj. 



k=l 



(73) 



But since ^(O) = ^j(Lj) = from (l56l l. the second term in (l73l vanishes. Replacing the remaining 
terms in (l72l . then 

o v k - 1 

x *$>j(xj)dxj = 0, 
(74) 



which must remain valid for all ^j(xj) such that ^j(xj) satisfies (|55T ) and (l56l ). A family of solutions 
for equation d74T l can be supplied by simultaneously noting from (l56l ) that 

e-t* x *il)j(xj) dxj (75) 
- r *,(•'•,.) • £? / e-teiVjlxj) dxj (76) 
= / ' < - ' ; !•'•, ) <**i, (77) 







e 



o 



where (a) is true since ^(O) = ^j(Lj) = by (l55l) . Supported by (1771 ) and from the fundamental 
lemma of the calculus of variations, we subsequently conclude that d74l ) holds only if the term inside the 
bracket is in the form of Kj exp(-Qxj) for some constant Kj. Thus, as a necessary condition, we have 

Pj (xj )p'j (xj ) - Ej ^^p^) + ( Xj ~ ^Pj( x j)) 

x dE MEtL lPk M)] + CMr{ ^ Pk{xk))] + K = o. (78) 



Remark 5: Rewriting (1781 ) as 



Pjixj) = [ - Pjixj) d 2 Pj (xj) J " j ~ t - y/ ' y( - r ' )) 



aEj ' r S i f' t) " +WHEwW)] + if J ]. P9) 



it is easy to verify that provides a degree of freedom to set the initial slope Pj( x j)\ x .- + of the 
power policy pj(xj). 

Now since the choice of j*' 1 coordinate was arbitrary, (178) holds for all j E [M]. Accordingly, we 
obtain a system of PIDEs with 2M degree of freedoms! In the following, we consider solutions in the 
infinite and finite battery cases: 

1 ) Infinite Storage Battery: Motivated by the converse result for the infinite storage battery, we consider 
a set of admissible policies as below 

(Afc/Cfc) +Q x k >0 

i>i, (-n. ) - \ (80) 
x k = 0, 

In the system of equations, {pfe(0 + )}j!Li and {p' k (0 + )}^L 1 (or equivalently {pk(0 + )}kL 1 and {Kk}t=i) are f ree parameters. 



where the excess power g > is added to ensure the positive recurrence of the process. In the limit, as 
g — > 0, the suggested policies in (l80b satisfy (|78T ) for all j, provided 



M X 



fc = l 

The average transmission power of (IBTfl l can then be evaluated as 



(81) 



npk(xk)} = ir k Pk(0) + {(\k/(k) + Q) / fk(x k )dx k (82) 

= ((A*/Cfc) + e)(l-Tfc) (83) 

On the other hand, in light of (l35l l we have E[pfc(xfe)] = Xk/Ck- As a result, a transmission node that 
exploits pk(xk) as transmission power policy has the following probability mass at zero 

< = (x 7FYZ ' (84) 

{Xk/Ck) + Q 

i.e., it sends information for a fraction e +^/cl ) °^ ^me. Moreover, associated with pk(xk), the mean 
square deviation of transmission power is given by 



a 2 (p k (Xk)) = {-±) 2 ir° k + / (pk(xk) - -±) fk(xk) dx k (85) 

= (A fc /Cfc)^. (86) 
For these power transmission strategies, we also have 

K > r(f)(V& + <?)) fl 7r¥^' ( 87 ) 

where the inequality follows from neglecting situations in which a subset of nodes are transmitting and 
the rest are silent due to battery exhaustion. As g J. 0, the upper bound (l36l l and the lower bound (l87T i 
coincide with each other. The total average throughput is given asymptotically by 

M 

TZ = r(^ Xk/Ck). (88) 

k=l 

Thus, near optimal performance of the energy harvesting system can be achieved when g I 0, and the 
behaviour of a classical communication systems (in the sense of using a constant power supply without 
interruption) can be closely approximated at the same time. 



2) Finite Storage Battery: In contrast to the case of batteries with infinite capacity, the system of 
equations in (P78T ) doesn't appear to admit a closed form expression for power policies when the storage 
capacity is finite. The remaining option is thus to solve (T78T ) numerically. However, the complexity in 
dealing with such systems is that the equations are not independent, but coupled. Alternatively, if all but 
one of the p k (-) are known, the remaining one can be obtained by solving a first order non-linear ODE 
in terms of the corresponding coordinate, using (178V First from (|47T ) and since G' k {x^) = g k (x k ) we 
obtain by integration 

\nG k {x k ) - lnG fc (0) = / * — ^- dv k . (89) 

Jo+ Pk{Vk) 

Therefore, 



x k \ 



Pk(v) 

By differentiating both sides with respect to x k , we obtain 



G k (x k ) = n° k exp ( / — ^ dv ) . (90) 



or equivalently, 



9k(x k ) = ' ) fc exp ( / —jL-dv), (91) 
Pk{Xk) Wo+ Pk{v) 



fk(x k ) = ir f , exp(/ -^r<fo). (92) 



Also due to the normalization condition 

4 + / fk(xk) dx k = 1, (93) 



we have 
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1 + / - — ? — ^ exp f / — ^— ) dx fe 
0+ Pfc(^fc) Wo+ Pfe(u) 



(94) 



which can simply be derived via substituting (l92l in (l93l and solving for 7r°. 

With the help of (l92l and (l94l) . we propose an iterative method, outlined as Algorithm 1, that computes 
a solution for (|78T ). Also, the convergence analysis of Algorithm 1 follows three steps 

i) at each iteration of Algorithm 1, the utility function (l38l l is non-decreasing, 

if) the utility function in d38l ) is bounded above, 

iii) Algorithm 1 thus converges if it is allowed to iterate indefinitely {i.e. no termination constraint). 

Specifically, in the first step, we denote the utility 1Z as an explicit function of the power policies in 
Algorithm 1, e.g., TZ(p{ (xi),P2 (x^),--- ,Pm( x m)) is the initial utility. After the A^th full iteration 
of steps 5-10 (outer loop) of Algorithm 1, in the jth iteration of 6-10 (inner loop), we then obtain 

P [ f +1 \xj) = axgmaxK(p[ N+1 \xi),--- ,p\*i 1 \x j -i),£,pfll(x j+ i), ■ ■ ■ ,p^\x M ))- 



Algorithm 1 



l: for all k 6 [M] do 

2: initialize Pk{xk) with some arbitrary function; 
3: compute d92]) and d94l ); 

4: end for 

5: repeat 

6: forj^l,Mdo 
7: calculate (1671 ); 

8: update Pj(xj) by solving (1781) for optimized values of j»j(0 + ) and K^; 

9: update d92]) and d94]) for k = j; 

10: end for 

11: until termination criterion is satisfied. 



Therefore, 

K( P [ N+l \ Xl ),... ,pf\ Xj ),... J a T(xm)) <K( P [ N+ V (xi)r .. ,pf^\ Xj ),... ,pVp( XM )). 

(95) 

In addition, since the objective function is upper bounded by d29l , we further have 

TZ sup = sup TZ{pi(x 1 ),p 2 (x2), ■ ■ ■ ,Pm(xm)) (96) 

{Pk(x k )} 

M A 

<r{^-^(l-exp(-( k L k ))). (97) 

fc=i U 

Concluding from (|95l and (|9Tb . the sequence 7^({p^ (xfc)}^ =0 ) must converge in the limit as N — > oo. 

Now, we concentrate on two important degenerate cases of our problem that substantially reduce the 
computational burden of solving the PIEDs. In the first scenario, suppose that all the transmission nodes 
scavenge energy in the same manner. By this statement, we mean that the statistical parameters of all 
the energy harvesters are identical, i.e., = A and Cfe = C f° r all A; = 1, 2, • • • , M. In the symmetric 
case, further assume that the batteries have identical capacities (L k = L). These assumptions then let us 
apply one policy to all the transmitters (pk{xk) = p{xk)) and to specialize (1781 ) as below 

,<9 2 % KEf = i P(* fe ))] 

d 2 p{xj 

x g^fe^ + C%KX>(^))] +K = 0, (98) 



where j is arbitrary and chosen from [M], and the operator Ey[-] now simplifies as 

M ,. M 

E j [r(Y / p(x k ))}= r(^p(x k )) ]J ir(dx k ). 

k=l JA 3 k = l ke[M]-j 

If we rearrange the terms in equation (l98l l. we have that 

p(xj) = &(p(Xj)), 
where the mapping : C 1 (0,L] — > C 1 (0,L] is given by 

fXj M 

&{p{xj)) =p(0+)- / fc + C%[r(^pK))] + (A-CK^)) 



(99) 



(100) 



fe=i 



dp{vj 



p( Vj 



d 2 p{ V] ) 



(101) 



As a result, it follows that the desired p(xj) is a fixed point for This then supplies us with an 

alternative algorithm for this special case (see Algorithm 2). 

Now in the second scenario, consider that there is only one transmitter in the communication system 
{i.e. a point-to-point setup). We thus have a simplified formulation as a necessary condition here, i.e., 



(102) 



As argued in [12], this is a second order, non-linear, autonomous ODE that can be solved numerically 
by employing linear multistep methods (e.g. Runge-Kutta or Adams-Bashforfh). The next lemma demon- 
strates some properties of solutions to this ODE: 
Lemma 1: Suppose K > — £r(A/C) in (11021 ). then 

(a) any solution p(x) is a strictly increasing function of x for p(x) > and p(x) — > oo as x — > oo, 

(b) p(x) grows doubly exponentially fast as x — > oo. □ 
Proof: Solving dl02| ) for p'(x), we have 

(A — £p(x))r' (p(x)) + C r {p{ x )) + K 



p'{x) 



(103) 



— p(x)r"[p(x)) 

From concavity of the rate function, r"(p(x)) < 0, the denominator is always positive. Thus, p'(x) > 
for all x > iff 



K > -[(\-(p(x))r'(p(x)) +(r(p(x))],Vx > 0. 



(104) 



Moreover, it is not hard to see that 

d 



dp(x) 



(A - (p(x))r' (p(x)) + £r(p(x)) 



2 In 2N, 



(p(x) - A 



(l+p(x)/N y 



Algorithm 2 



1: initialize p(°\xj) with some function. 
2: repeat 

3: compute ([92]) and (194k 
4: compute d99l ; 

5: update p( JV+1 )(x J ) = ^(p( JV )(x J )) from (fTOTT) for optimized values of p( N \0+) and K^; 
6: until termination criterion is satisfied. 



Hence, p(x) = X/C, is a global maxima for the right hand side of (11041 . Replacing p(x) = X/( into 
(11041 ). the numerator of (11031 ) is then lower bounded by 

[K-(r(\/C)]>0. 

Additionally, we upper bound the denominator by 

-p(x)r (p(x)) = — — 

21n2(l+p(x)/7V ) 

< l/No 
~ 8 In 2' 

Therefore, 

and thus p{x) — > +oo. To prove the second part of Lemma [T] consider the substitution 

p(x)/N = exp(S(x)), (105) 
where S(x) increases since p(x) increases. Then we have 

S(x) — > +oo, as x — s- oo. (106) 



Consequently, 



dr{p(x)) 1 1/jVo 1/JVo , Q( nn „. 

— . / \ = — : V^r-; — — — exp — (107) 

21n2 1+p(x)/iVo 21n2 PV v 



d 2 r(p(x)) _ -1 l/iV 2 -1/JV, 



dp(x) 2 2 In 2 (l + p(x)/N ) 2 2 In 2 



exp(-25(x)), (108) 



and 



r{p(x)) = ilog 2 (l +p(x)/iV ) ~ gl^^)- (109) 



Replacing (1107l) - dl09l ) in (11021 ) yields that for x -> oo 



S'(x) + (A - CA r oe 5(a;) )(e- 5(a;) /^o) + + K = 



(110) 



with if = K/21n2. Thus, as x ->■ oo, (11061 ) reduces (II lot to 



C5(x) = 



(111) 



which has the following solution 



5(x) = y4exp(Cx) 



(112) 



for some constant A, and thus 



p(x) = 0(exp(e^)) 



as x — > oo. 



(113) 



Remark 6: On account of Lemma 1, it is easy to verify that when K < — £r(A/£), the property of 
(II 131 ) does not hold in general. In fact, for sufficiently large negative K, solutions of (11021) are non- 
increasing. In the succeeding section, we will discuss that a consequence of this condition is suboptimal 
power policies for transmission. 



We now study a multiple access communication system consisting of two nodes (M = 2) with Ai = 
A2 = A = 1 and ("1 = C2 = C = 1- Because of the symmetry of the MAC, the achievable power policies 
for this setting are obtained through Algorithm 2. Nevertheless, to implement Algorithm 2 according to 
steps 1-6, one is obliged to search for optimized values of p(0 + ) and K at each iteration. To ease this 
process and in what follows, Algorithm 2 is modified in a way that once the values of p(0 + ) and K are 
initialized, the same values are used at each iteration step. 

With this modification, Fig. [Ta| then illustrates the designed power policy as a function of the remaining 
charge in the battery with initial conditions p(0 + ) = 0.1 and K = and initializing function p^°\xk) = 
%k +p(0 + ), < Xfc < Lfc. After N = 10 iterations, the power policy has converged to a solution of (|98l ). 
It can also be seen from Fig. [la] that as the remaining charge in the battery increases, the transmission 
power also increases rapidly. Supported by part (b) of Lemma [TJ we further conjecture that this increase 



V. Discussion and Numerical Solution 



Although this approach is potentially suboptimal, it always yields achievable results, and in the case of the considered example 
here, the achievable results are close to the upper bound. 



TABLE I: 

Total average throughput for two identical nodes, using Shannon rate function, r(x) = |log(l + x/Nq), with 
Ao = 1, equation constant K = 0, initializing function p(x) = x + p(0 + ),0 < x < L, and for various storage 
capacity L and initial values p(0 + ). 



Initial Value p(0 + ) 



L 


-Supper 


0.001 


0.001 


0.1 


1 


0.5 


0.4187 


0.3177 


0.3152 


0.3094 


0.2797 


1 


0.5895 


0.4217 


0.4159 


0.4069 


0.3722 


2 


0.7243 


0.4634 


0.4575 


0.4511 


0.4075 


3 


0.7681 


0.4652 


0.4593 


0.4510 


0.4091 



TABLE IT: 

Total average throughput for two identical nodes, using Shannon rate function, r(x) = |log(l + x/Nq), with 
Aq = 1, initial value p(0 + ) — 0.001, initializing function p(x) = x +p(0 + ),0 < x < L, and for various storage 
capacity L and equation constant K. The upper bound for an infinite storage battery (Lk — oo) is given by 
Roo = |log(l + 2) = 0.792. 

Equation Constant K 
L i?u PP e, +0.5 -0.5 Optimum(% of JJupper) 

0.5 0.4187 0.3017 0.3177 0.3057 0.3262 (77.9%) [K=-0.15] 

1 0.5895 0.3707 0.4217 0.4410 0.4612 (78.2%) [K=-0.37] 

2 0.7243 0.3854 0.4634 0.5725 0.5951 (82.1%) [K=-0.63] 

3 0.7681 0.3858 0.4652 0.5907 0.6654 (86.6%) [K=-0.67] 



is in fact doubly exponential in x. Indeed, when the occupied capacity of the battery becomes large, the 
chance of overflow due to new energy arrivals increases as well. In this regard, an optimal power policy 
is one which consumes the battery charge fast enough such that the occurrence of overflow is traded-off 
against suboptimality of employing a large instantaneous transmission power (see Section IV). On the 
other hand, for sufficiently large negative K, solutions of d98l ) are non-increasing (ref . Remark [6] for the 
point-to-point case) and they thus fail to manage battery overflow. In view of this issue, here we are only 
concerned with increasing power policies. 

Corresponding to the designed power policy in Fig. [la] Fig. [Tb] shows the absolutely continuous part 
(density) of the probability measure in d8j. In this case, consistent with our earlier observation for the 
power policy, the density function also falls off quickly. In terms of ergodicity, this is basically an assertion 
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Fig. 1: Battery capacity L = 3, equation constant .JT = 0, and p(Q + ) = 0.1 for two nodes case, (a) The convergence 
of transmission power policy to an achievable policy (denoted by squares) after N = 10 iterations with initializing 
function p^(x) = x+p(0 + ) (dashed lines) and iterates (solid lines), (b) Absolutely continuous part f(x) of tt°(x) 
for the converged solution. 



of the fact that the system spend little time with large stored charge in the battery. Using Algorithm 2, 
we have computed the achievable rates provided in Table I and Table II, where the termination criterion 
is taken to be 



r(ZtiP N+1 (xk))-r(ZtiP N (*k)) 



EjLiP^O"*)) 



< l c 




o 1 2 :i 

Battery charge x 

Fig. 2: Transmission power policies p(x) with different initial values (L = 3, M = 2, K = 0). 




Battery charge x 



Fig. 3: Robustness to the initializing function, using a constant initializing function (dashed line) p(°\xk) — 
p(0+), < x k < L {L = 3, M = 2, K = and N = 10). 

i.e., the iteration stops whenever the increase in rate is less than one percent. With this precision, Table 
U shows the total channel throughput for several choices of p(0 + ) and fixed K = 0. The upper bound is 
also computed from d29l ) and denoted by i? U pper in the table. 

Based on a comparison between the upper and lower limits on the rate function, it is immediate that 
the choice of p(0 + ) = 0.001 results the best performance of the designed power policy. For the same 
choices of p(0 + ) as in Table U and K = 0, Fig. |2] shows the power policy solutions. Except for the case 



of p(0 + ) = 1, all the power policy solutions adopt a small transmission power when battery charge is 
small. Along the same lines, Table [HI shows the upper and lower limits on the average throughput for 
fixed p(0 + ) = 0.001 and variable K. We have particularly provided the best value of K up to precision 
0.01 as well as the corresponding achievable rates. The best achievable rate, as a percentage of the upper 
bound, is also evaluated. 

Finally, to show the robustness of the iterative algorithm to the initializing function, a different choice 
of p®(xk) is studied in Fig. [3] Therein, we particularly have selected p®(xk) = pk{0 + ),0 < Xk < 3 
for purpose of initialization in Algorithm 2 while the rest of the parameters are the same as in Fig. la. 
Evidently, the power policy converges to an identical function as one depicted in Fig. la. Similarly, the 
same convergence was observed when p° k {xk) = Pk(0 + ) + y/x~k, < Xk < 3. In this respect, the proposed 
algorithm appears to be insensitive to the choice of the initial power policy. 

VI. CONCLUSION 

We have considered continuous-time power policies for a multiple access communication system where 
each node is capable of harvesting energy. First we modelled the battery as a compound Poisson dam, 
where the remaining charge in the battery modulates the transmission power. We then analysed this 
storage dam model in the ergodic case. In particular, we characterized an upper bound on the maximum 
sum-rate as a function of the energy arrivals distributions and the capacity of the batteries. For batteries 
with infinite capacity, we proved that any rate close to this bound is achievable by a set of constant 
power policies. For batteries with limited capacity, we showed that optimal power policies can be derived 
by solving a system of simultaneous partial integro-differential equations. To solve these equations, we 
developed an iterative algorithm based on the Gauss-Seidel approach. We next specialized this algorithm 
for the symmetric MAC case where the multiple access nodes have identical energy harvesting statistics. 
Furthermore, by showing the concavity of the throughput maximization problem, the convergence of the 
proposed algorithms was established. Numerical results show that for L = 3, the achievable scheme 
provides throughput up to 86.6% of the upper bound. 

Potential future work includes extending the study to the transmitters with vacation times. This could 
model the sleeping state of sensors in which a sensor stops sending information for a random time 
duration after exhausting its energy (see ||7]). 

Appendix A 

In the following, we show that for every power policy with memory, p* k {Xk{u);u < t), there exist 
a memoryless counterpart pk(Xk(t)) that attains the same or better channel throughput performance. In 



particular, let (CI, F , P) be a complete probability space and X k (t;w), — oo < t < oo be a stationary and 
ergodic stochastic process denned on this probability space and whose evolution for t > is given bj 



X* k (t;w)=X* k (0;w) + 4 Q ((0, t];w) - / p* k (X* k (u;w);u < s) ds. (114) 

Jo 

In conjunction with the process X k (t;w), we then define the following empirical CDFs, 

F k (p k ,x k ;w) = lim — / 1 (p k (Xl(u; w); u < s) < p k ) l(X k (s; w) < x k ) ds, (115) 

1 f T 

lim - / l(X* k (s;w) <x k ) ds (116) 



7r fc (x fc ;u;) = t 



lim F k (p k ,x k ;w), (117) 



and the conditional CDF, 

?*/ i \ F k (p k ,x k ;w) 

F k (p k \x k ;w) = — , (118) 

TT* k (x k ;w) 

if TT* k (x k ;w) > 0. In the case of x k such that TT k (x k ;w) = 0, the conditional CDF F k (p k \x k ;w) 
is arbitrary. Now since X k (t) is ergodic, F k (p k , x k ;w) and K k (x k ;w) are well defined, and constant 
P— almost surely on Cl, i.e., 

F* k (p k ,x k ;w) V =* F* k (p k ,x k ), (119) 
7r* k (x k ; w) ¥ = s TT* k (x k ), (120) 

and 

F k (p k \x k ;w) F = s F k (p k \x k ), (121) 

for some functions F k (p k ,x k ), ir k (x k ), and F k (p k \x k ). Next, we define the memoryless power policy 
p k (x k ) as follows 

f'OO 

Pk(x k ) = / p k F k (dp k \x k ), (122) 
Jo 

and a corresponding storage process X k (t) governed by 

X k (t) = X k (0) + 4 n (0,t}- [ p k (X k (s)) ds. (123) 

For clarity, we make the dependence on w G Q explicit. 



Also, we denote the stationary measure of -X&(i) by nk{xk). Our objective now is to prove that the 
throughput using the storage process with memory p* k {Xl{u;w),u < t) is no better than that of it's 
memory less counterpart p k (X k (t)), i.e., 

K({pUX* k (s;wy iS < t)}tU) V Tn{{p k (X k {t))}kU). (124) 

To show this result, we begin with the definition of the long term average throughput for the storage 
process in (II 14b . i.e., 

i f T m 

n({pUX* k (s;w);s<t)} A k Li) = ton = / r^^( S ;H S <t) ds (125) 

= / / r{Y^ Pk )T\F* k {d Pk ,dx k -w) (126) 
J * J B k=1 fe=i 

P = S / / r{Y^ Pk )T\F* k {dp k ,dx k ) (127) 

k=i k=i 

„ „ A/ M 



k=l k=l 



where (a) follows from the definition of F k (dp k ,dx k ;w) in d 1 15b . and B = [0, oo) x [0, oo) x • • • [0, oo) 
is the domain of integration on {p k } k Li- From concavity of the rate function, we then upper bound (11281 ) 
as follows 



M M M 



^(K(X fc *( S ;u;); S <t)}f =1 ) P = S f [ r(f2f>h)f[Fttdpi\xt)f[ir* k (dx h ) (129) 

r M „ M M 

< / r (j2 / ^n^^i^n^c^*) (i3o) 



A k=l B 1=1 k=l 
M coo M 



A 



fc^ it=i 

„ AT A/ 

/ r{J2Pk(xk))]jK(dxk), (132) 



fc=i fc=i 

where the last step follows from the definition of p k (Xk) in (1 122b - The remaining task is now to show that 
n k( x k) = 7Tfe(^fc)iVA; G [M]. For this purpose, we define some notation in conjunction with an arbitrary, 
stationary, cadlag^ process Y(t) whose jumps (positive or negative) occur at time instants T°, T 1 , ■ ■ ■ . In 
particular, the right hand derivative of Y(t) is defined by 

Y^^mui^m. (13 3) 

Right continuous with left hand limit. Note that both the storage processes in i ll 14) and d 1 23t are cadlag. 



In addition, define 



Y(t~) = limY{t-e). (134) 



Theorem 2: Let Y(t) be an ergodic, stationary, cadlag process. Then, 

f(y)E[Y+(t)\Y + (t)=y] = A°E° [l {Y{TO ,- )>y} l {Y(T o )<y} - l {y(T o,- )<?/} l {y(T o )>y} ] , (135) 

where f(y) is the probability density at y, and E° denotes the expectation with respect to the Palm 
probability distribution corresponding to the point process (with assumed intensity A ) for the jumps. □ 
Proof: A proof based on the rate conservation law (RCL) can be found in [11, pp. 36]. ■ 

Remark 7: The term l{Y(T°<-)>y}l{Y(T°)<y} m the right hand side of Theorem [2] corresponds to 
negative jumps in the sample path while l^ Y (T '-)<y}^-{Y(T°)>y} corresponds to positive jumps. 

Remark 8: As the memoryless storage process in (11231 only contains positive jumps, 

E °[ 1 {^(^--)>x fc } 1 {X fc (TO)<a ;fc }] = 0, 

where as defined in Section HUB, Tjj, T^., ■ ■ ■ denote the energy arrival times for the k th node. In this 
special case, we then have 

f k (x k )E[X+(t)\X k (t) = x k ) = f k (x k )E[-p k (X k (t))\X k (t) = x k ] (136) 

= -fk{xk)Pk{x k ). (137) 

For the right hand side of Theorem [2] we obtain 

\ E l-l {Xk (T°,-)<x k }l{X k (T°)>x k }] = >>kEl-l{X k (T°,-)<x k }l{X k (T°)>x k }} (138) 

= -A fc / (1 - B k (x k - v k ))ir k (dv k ) (139) 
Jo 

(l - B k (x k ))ir k + / (l -B k (x k - v k ))f(v k ) dv k , 

(140) 

where (a) follows from the notion of Poisson Arrivals See Time Averages (PASTA) [15, Prop. 1.23] for 
Poisson energy arrival process with intensity A = X k . Equating (11371 ) and (11401 ) according to Theorem 
|2j we obtain 



r f Xk 
fk{xk)Pk{xk) = A fc (l - B k (x k ))7rl + (1 - B k (x k - v k ))f k (v k ) dv k 



(141) 



which is the equilibrium condition in (fTOl with the density f k (x k ) and the atom 7r°. 



Returning to the storage process with memory in (I114K now it is also easy to see that 

poo 

E[(X* k (t))+\X* k (t) = x k ] = p k F k (dp k \x k ) (142) 



o 



= Pk(x k ), (143) 

which is simply the average rate of down crossing at level x k corresponding to the stationary distribution 
of X k (t) in (I114I ). Applying the same technique as Remark [8] to the process in (11141 ). we thus obtain 

r k ( Xk )n(x* k (t)) + \xi{t) = Xk ] = r k {x k ) Pk {x k ) a44) 



(l - B k (x k ))n* k ' + I (l - B k (x k - v k ))f k (v k ) dv k 

(145) 

where Tr k '° and f k {x k ) are the atom and the continuous part (density) of the probability measure TT k {x k ). 
Since from Theorem 1 the probability measure that solves (11411 ) and (11451 ) is unique, 

7T* k (x k ) = 7T k (x k ), yx k . (146) 

Concluding from (11321 . we thus showed that 

P— as r JH- 

K{{p* k {Xl(s]w);s <t)} k ! =l ) < / r( y2p k (x k ))7rl(dx k ) 

J A 7~, 



k=l 

M 

r 



^Pkjxk^^kjdxk) 
K({p k (X k (t))}tU) 



A k=i 



Since for all k ^ j, 



we have that 
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(^k' a i9k(xk)) = (nl^^Kxk)) = (ir° k ' 2 ,gl(x k )), 



iT k (dx k ) = irl(dx k ) = 7rl(dx k ). 



Then, 

ft? = 



A 



r A 



* 3 <??(^) 



\- , G k (x k ) \ 



fce[M]-j 



j7r?(dxj) J| TT k (dx k ) 
ke[M]-j 



r A 



G?(x,) 



E * 



Gfc(x fc )\ 



7r?(dx,) 



Eh 



r A 



fee [AT] -j 



, :[ , ;] ; 



L-/0 



r A 



O.Qtiti 



E A 

fee[M]-j 



(147) 
(148) 

(149) 

G k {xk) V 
' 9k(x k ) J _ 

(150) 



For the term inside the first expectation in (I1501 l. we proceed as below 



r A 



+ , ^kGk(xk) 



r A 



agj( Xj ) + ag 2 -(x 3 



"m /•' / cm f;!(.r,) ^ A/,r//,i.r/,) 



> 



+ 



r A, 



y 3 y v ke[M]-j 



( or -'• '' c/-'(./-, 
r A 



a 



r A 



ae g 2 (xj) 
G](x 



+ E A 



ae < ' jX - i g^(xj) dxj 



E 

ke[M]-j 



k€[M]-j 



J 9j(xj) ' ,_Jrzi , gk(xk) 



r X 



G 2 (x j/ 



AfcGfc(xfc) 



E 



e ^ Xj gj( X j) dxj 



where (a) can be verified via lemma given in Appendix C and choosing 

0l = ae-^'^^Zj), a 2 = ae //;(.'•,). 



6i n< c -''(;;(.r / ). 



6 2 = ae-^G5(x i ), 



and 



7 = A,, /? = £ 



fce[M]-j 



AfcCfc(xfc) 

fffc(Xfc) 



Therefore, for the first term of (11 501 we obtain 

-Li 



E, 



(, G K x i) , \- , G k (x k )\ _, x . 



> aE 



r A 



ke[M]-j 



r A 



Splitting the second term of (I150l l as 



7T/ Ej 



■fce[M]-j 



OTT/ Ej 



fce[M]-j 



G k {x k ) \ 
' 9k(x k ) J 



and combining (11511 ) and (11521 ) we derive 



i - 0,2^ 
+ OCTTj Ej 



(151) 



E a, 

ke[M]-j 



G k (x k ) 
' 9k(xk) 

(152) 



Kf > all] +aR 2 r 



(153) 



Appendix C 



Lemma 2: Let 7, /3 > 0, > and 6;. > be given. Then 

Va fc r( 7 -^ + /3)<ar( 7 - + /3), 



(154) 



where a = J2 k a k and b = J2 k b k- □ 
Proof: we define the function V(x) = xrU'y/x) + /3) which is known to be concave for all x > 
since 

2 

V"(x) = \r"(l + P) < 0, 

where the concavity property of the rate function has been used. We then proceed as 

£ a k r(^ + /?) = £ h(a k /b k )r(^ + f3) 
k ak k 



y~]b k V(a k /b k ) 

k 

bY,(h/b)V(a k /b k ). 



Furthermore, from concavity of V(x), 

bJ2(h/b)V(a k /b k ) < bV(J2h/b x a k /b k ) 
k k 

= bV{a/b) 
= ar {l^ + P)- 

Hence, 

Va fc r( 7 -^ + /3) < ar( 7 - + /3). 
, a k a 
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